🕷️️ Job Radar • SCRAPING

freelancer.com 🟡 2026-05-19

🔹 Automate Data Scraping and Post-Processing with Python and Java
👤 Client: 🇪🇬 Cairo, Egypt Member since 2024-09-17
💰 Price: $20 / hr Average bid
🚩 Problem: Need a seamless integration between Python scraping tools and Java orchestration for reliable product detail extraction and automated post-processing.
📦 Existing: [Stack]

Specifications:

[Target] Product details (titles, prices, SKUs, images, stock status)
[Method] Python (requests, BeautifulSoup, Scrapy, Selenium) for scraping; Spring Boot for orchestration
[UI/UX] N/A
[Stack] Python: requests, BeautifulSoup, Scrapy, Selenium; Java: Spring Boot, PostgreSQL
[Security] Basic authentication and encryption for sensitive data handling
[Format] JSON/CSV output from scrapers; PostgreSQL database storage

Workflow:

1. Develop modular Python scrapers using Scrapy or custom scripts with BeautifulSoup.
2. Integrate scrapers to run from the command line, ensuring error-free execution.
3. Design a Java-based automation module leveraging Spring Boot for scheduling and processing scraped data.
4. Persist scraped data in PostgreSQL database.
5. Implement logging for success/failure events of each scheduled job.